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observation. Only with the advent of the telescope 

could astronomers collect the data that led to theories 
of planetary motion and gravitation. We are witnessing 
astonishing improvements in medical imaging, revealing the 
untold secrets of biology and disease (page S82). This Outlook 
examines imaging from two perspectives: developments in the 
technology itself, and applications of imaging to medicine. 

One major medical challenge is determining how the brain 
deteriorates in Alzheimer’s disease. Now, imaging is verifying 
that tangles of tau proteins are leading protagonists (S84). 
Inflammation is at the core of diverse disorders, from 
atherosclerosis to cancer, and it is becoming clear that defects 
in protein clusters known as inflammasomes are primarily 
to blame. Fluorescent microscopy is starting to show how 
inflammasomes form and respond to stimuli, and might be 
worthy therapeutic targets (S86). 

Enhanced medical vision makes surgery sharper (S88). 
Imaging during a breast lumpectomy, for example, allows 
surgeons to remove the small malignant bits that are often left 
behind using conventional techniques. 

Medical technologists are working on several fronts to 
improve the quantity and quality of information gathered 
ina single scan. That often means combining modalities 
in complementary ways — for example, adding structural 
information from computed tomography to the functional 
data from positron emission tomography (S90). And smarter 
software can focus indistinct blurs (S96). Still, medical imaging 
is failing to reach its full potential, says Alan Moody. He argues 
for the creation of “big picture” a network that will enable 
researchers to readily access the vast library of clinical images 
that are typically used once and then stashed away (S95). 

We are pleased to acknowledge the financial support of 
Navidea Biopharmaceuticals in producing this Outlook. 

As always, Nature retains sole responsibility for all editorial 
content. 

Herb Brody 
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ENHANCED MEDICAL VISION 


The ability to look inside the human body without using a scalpel has revolutionized how 
we diagnose and treat illness and injury. By Brian Owens. 
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TYPES OF SCANS 
MRI 


Magnetic resonance imaging is used to 
examine soft tissue such as joint ligaments, 
muscles, and the brain. It uses powerful 
magnetic fields to manipulate the spin of 
protons in the tissue to provide information 


about how they are arranged. 
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CT 


Computed tomography uses 
computers to process X-rays ard 
create images of slices of the body. 
These cross-sections can be 
assembled into larger 3D images to 
detect tumours, bone damage, and 
potential hemorrhages, as in this scan 
of colonic diverticular disease. 


ULTRASOUND 


Sound pressure waves with a frequency 
beyond human nearing can be sea ta 
image soft tissues in the body, most 
commonly a developing foetus. 


MEDICAL IMAGING Mietuumetes 


RISE OF THE MACHINES RISKY BUSINESS 

There has been a dramatic rise in the number of CT and Some imaging procedures can expose patients to high 

MRI scans over the past decade in the United States. levels of radiation, so the number and timing of scans 
must be carefully controlled over their lifetime. 
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X-RAY fMRI 


The oldest and still most common Positron emission tomography detects the A research technique that is beginning to 
form of medical imaging is used to gamma rays produced by a radioactive tracer find clinical uses, functional magnetic 
examine the lungs and hard tissues introduced into the body that is specific to the resonance imaging measures brain 
such as bones. tissue of interest. It is used extensively in activity by detecting associated changes in 
cancer diagnosis, to locate tumours such as blood flow. 
this one in the lung. 
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UTR MEDICAL IMAGING 


Orange and blue indicate deterioration in the brain of a 65-year-old Alzheimer’s patient. 


ALZHEIMER’S DISEASE 


Mapping the 
brain’s decline 


Imaging the brains of Alzheimer’s patients provides 
insights into the way this insidious disease progresses. 


BY SARAH €. P. WILLIAMS 


ll the memory and cognition tests 
Aimee that the 87-year-old man had 

Alzheimer’s disease, and his brain scan 
lit up with yellows and reds. The colours were 
thought to show the presence of tau tangles, the 
clumps of proteins in the brain that have been 
linked with Alzheimer’s, but the accuracy of 


the scan had never been verified in humans. 
So it was filed away. Two weeks later, the man 
— a participant in a study by Philadelphia, 
Pennsylvania-based Avid Radiopharmaceuti- 
cals — died from an unrelated cause. When his 
brain was autopsied, researchers found not only 
amyloid plaques, giving a definitive diagnosis 
of Alzheimer’s disease, but tau tangles in all 
the same places that lit up in the scan. For the 
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first time, the accuracy of the scan — a form of 
in-vivo tau imaging — had been confirmed. 

Tau imaging isn’t the only method being 
developed to visualize the effects of Alz- 
heimer’s disease on the brain, but it’s one of 
the most recent. As these imaging methods 
improve, researchers hope they will help reveal 
the underlying causes of Alzheimer’s disease, 
which affects more than 35 million people 
around the world. Clinicians are hoping the 
scans will allow them to diagnose patients 
earlier. And pharmaceutical companies are 
aiming to develop better drugs, and conduct 
shorter and smaller clinical trials. 

“The point of new imaging is to give you a 
time-lapse movie of the disease and then see how 
different interventions change that movie,’ says 
neurologist Paul Thompson of the University of 
California, Los Angeles. “You need to be able to 
track the spread of the disease in more ways than 
asking people memory questions” 

Scientists know that patients with Alzhei- 
mer’s disease have a progressive loss of cells in 
certain areas of the brain, and an increase in 
two types of protein: amyloid-B, which accu- 
mulates to form amyloid plaques; and hyper- 
phosphorylated tau, which forms tangles. But 
whether tau or amyloid build-ups are causes or 
effects of the disease is not known. And what 
other factors are involved remains unclear. 

Historically, a tentative diagnosis of Alzhei- 
mer’s follows a battery of cognitive tests. A defin- 
itive diagnosis is only achieved post mortem 
with an autopsy of the brain — a procedure 
rarely done outside research studies. Since the 
mid-1990s, clinicians have also been using mag- 
netic resonance imaging (MRI), which shows 
the loss of brain cells associated with the disease. 
These scans have shown that the brain begins to 
change years before symptoms appear. 

Nick Fox, a neurologist at University Col- 
lege London, was one of the first to report the 
changes revealed by MRI up to a decade before 
the onset of symptoms. For the past ten years 
he has been involved in a longitudinal study 
of people with an inherited predisposition for 
Alzheimer’s disease as he explores the early 
changes to the brain. Comparing MRI scans 
of individuals over time has allowed him to see 
which parts of their brain begin to shrink first, 
and when’. “There’sa long period of time when 
people have Alzheimer’s disease but typically 
aren't diagnosed,” Fox says. 

The MRI scans show that the death of brain 
cells precedes Alzheimer’s symptoms by five 
or six years. The goal of newer imaging meth- 
ods is to detect these changes even earlier, and 
more precisely track disease progression. 


BRIGHT SPOTS 

Tau imaging uses an experimental form of posi- 
tron emission tomography (PET). Before the 
scan, clinicians inject the patient with a tracer 
molecule called T808 that attaches to any tau 
protein it encounters — the tracer is what caused 
the bright spots of yellow and red in the Avid 
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Radiopharmaceuticals study. But the first PET 
tracer designed for Alzheimer’s disease dates 
from the early 2000s, when researchers at the 
University of Pittsburgh unveiled Pittsburgh 
compound B (PIB), which binds to amyloid’. 
Scientists around the world now use this tracer 
to track amyloid deposits in clinical trials and 
research studies. Pharmaceutical companies are 
looking for amyloid tracers that can be used in 
imaging studies in the general patient popula- 
tion, not just for research. 

The science behind the formation of tau tan- 
gles and amyloid plaques is still unclear, however, 
so not all scientists are convinced that amyloid 
tracers will be the best way to track the disease. 
Tau tracers, including T808, are being developed 
as alternatives, alongside a host of other types 
of scans. For example, a version of MRI called 
diffusion tensor imaging can provide detailed 
images of the connections between parts of the 
brain and reveal changes to its microstructure. 
Ina study published earlier this year, Fox’s team 
found that such microstructure changes were 
present in people at high risk of Alzheimer’s dis- 
ease, even when normal MRI could not detect 
larger structural losses’. 

Other researchers are turning to functional 
MRI, which shows the areas of the brain that 
are active during any given task, or when the 
brain is at rest. They have discovered that a 
task-free, or resting-state, functional MRI scan 
reveals alterations in pre-clinical disease, the 
stage before clinical symptoms’. “Even if you 
only have a basic MRI scanner, you can get a 
whole handful of these variations of MRI scans 
in only about 30 minutes,’ says Thompson. 


SEEING RESULTS SOONER 

Researchers working to develop scans see their 
work as a necessary step to improve the way 
drugs to treat Alzheimer’s are studied. “It takes 
too many participants and too much time to test 
preventive drugs for Alzheimer’s, says Eric Rei- 
man, executive director of the Banner Alzhei- 
mer’s Institute in Phoenix, Arizona. “We're all 
interested in developing faster ways to do this.’ 

There are many kinds of Alzheimer’s disease 
clinical trials, such as testing drugs or lifestyle 
interventions, with the aim of preventing dis- 
ease or treating symptomatic disease. But the 
primary endpoints are usually the same: meas- 
ures of cognition, or examinations of autopsied 
brains. This means that a trial, particularly if 
it begins before the onset of dementia, can last 
for decades before reaching an endpoint that 
shows whether a drug has been successful. 

If imaging techniques could provide results 
sooner, trials could be much less expensive, 
Reiman says. But the challenge is gathering 
enough evidence to show that the biomarkers 
— such as the amyloid or tau PET tracers — 
are a reliable substitute for measures of actual 
memory and cognition. “Its a catch-22. You need 
clinically proven treatments to prove that the bio- 
markers work,’ says Reiman. “But the biomarkers 
will help us develop new treatments much more 


readily and get them to patients sooner” 

One approach, he says, is to focus on popu- 
lations of people with genetic risk factors for 
Alzheimer’s disease. Testing interventions in 
such populations requires fewer participants to 
achieve statistical significance because a higher 
proportion will develop the disease. So Reiman 
and his colleagues at Banner have begun work- 
ing with the US National Institutes of Health 
and Genentech — a biotech subsidiary of 
F. Hoffmann-La Roche based in South San 
Francisco, California — to test an amyloid anti- 
body called crenezumab in an extended family 
in Colombia that has a genetic mutation lead- 
ing to early onset Alzheimer’s. As well as testing 
the drug, the scientists are using PET and MRI 


SLIPPING AWAY 


Positron emission tomography scans with a tracer 
(FDDNP) that binds to both amyloid plaques and 
tau tangles visibly correlate with cognitive decline. 
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scans with the latest tracers to track the disease 
in the participants. They have detected amy- 
loid plaques in family members from around 
28 years of age, almost two decades before the 
typical onset of disease in those without the 
mutation. 


GENES AND SCREENS 

The latest forms of brain imaging, which pro- 
vide increased precision and earlier glimpses 
of disease, along with the decreasing costs of 
well-established scans, are a boon to pharma- 
ceutical companies looking to test drugs. But 
they also offer basic researchers a way to find 
out what causes Alzheimer’s. 

At UCLA, Thompson heads Project 
ENIGMA, the world’s largest brain imag- 
ing study. Scientists based in 20 countries 
contribute all types of scans of their patients’ 
brains, along with information on the patients’ 
health and genetics. The network, which so far 
includes more than 26,000 brain images, pro- 
vides a way to do large-scale automated studies 
on the specific features of brains from people 
with Alzheimer’s disease or with genetic risk 


MEDICAL IMAGING #uuiaelels 


factors for Alzheimer’s. “When we pair imag- 
ing and genetics, we can screen the brains of 
certain gene carriers and compare them with 
non-carriers, Thompson explains. “Then we 
can ask: what's different about these brains?” 

When they looked at one gene known to bea 
risk factor for Alzheimer’s, CLU, the ENIGMA 
researchers discovered that a variant of the gene 
can damage wiring in the brain when a per- 
son is about 20 years old’. Thompson says that 
the variant of CLU, which encodes the protein 
clusterin, “doesn’t actually give you Alzheimer’s 
directly, but it gives your brain a punch that 
makes any second blow harder to deal with” 

More recently, researchers found another 
risk gene for Alzheimer’s, TREM2, that is 
prevalent in Iceland®. Within five months, 
Thompson's team had sorted through the 
ENIGMA scans to find entries that carried 
the gene variant. They were able to show, in 
an as yet unpublished study, that the TREM2 
variant speeds up brain cell loss once someone 
has developed Alzheimer’s disease. 

As such discoveries progress — linking 
genetics to structural and functional changes 
in the brain using automated, high-throughput 
methods — Thompson hopes that some com- 
mon molecular pathways will emerge that can 
help explain dementia. By seeing inside the 
brain, the scientists can get a clearer picture of 
what happens during the onset and progres- 
sion of Alzheimer’s. 

These latest brain scans are now widely used 
for Alzheimer’s clinical trials and research stud- 
ies, but they are far from routine in the clinic. 
A definitive, early diagnosis of the disease can 
give patients and their caregivers a better idea of 
the prognosis, but there isn’t enough evidence 
that the results of a scan will change the clini- 
cal outcome. It will take an ongoing interplay 
between better imaging and the development 
of treatments to change this, says radiologist 
Clifford Jack, an Alzheimer’s imaging special- 
ist at the Mayo Clinic in Rochester, Minne- 
sota. Both sides of the equation — scans with 
the ability to screen disease, and treatments 
that slow preclinical disease — will need to be 
developed first. “At some point, we will develop 
screening methods and early intervention 
treatments,” says Jack. “Just like with cardio- 
vascular disease today, we can identify people 
with high blood pressure and high cholesterol, 
which may not be provoking symptoms, and 
intervene.” Until then, improvements in brain 
imaging techniques will help scientists work- 
ing on Alzheimer's disease to better understand 
this devastating and deadly cognitive decline. m 


Sarah C. P. Williams is a freelance science 
writer based in Kailua, Hawaii. 
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One burning question: why is there only one inflammasome (red dot) per cell (nuclei stained blue)? 


INFLAMMATION 


A complex problem 


Multi-protein inflammasomes are being implicated in a surprising number of diseases, and 
researchers are keen to find out why. 


BY KATHARINE GAMMON 


r | The immune system’s primary task — 
telling friend from foe — is no easy job. 
Faced with a serious threat such as an 
infection or wound, the system needs to send 
defenders out immediately. But it also needs to 
lie low in the face of innocuous visitors. 

When confronted by a dangerous outsider, 
the human body has two lines of defence. 
There is learned immunity, which is acquired 
by exposure to a pathogen, either from the 
environment or through a vaccine. Then 
there is innate immunity, which is the imme- 
diate hard-wired reaction to outside invaders. 
The key ingredients in the innate response are 
inflammasomes — large protein complexes 
that form in response to a perceived threat, 
sounding the alarm for the body’s inflamma- 
tory responses. 

A growing body of research indicates 
that defects in the structure and activity of 
inflammasomes are central to a vast num- 
ber of illnesses, from atherosclerosis and 
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arthritis to Crohn’s disease, cancer, diabetes 
and irritable bowel disease. Targeting these 
complexes could usher in a new age in drug 
development. But doing so requires precision 
methods of seeing inflammasomes at work. 


SOUNDING THE ALARMIN 

Over the past decade, researchers have begun 
to piece together how inflammasomes work. 
These large molecular complexes form in 
response to different stimuli — and their com- 
position can differ accordingly. The stimuli can 
be incredibly diverse, including bacteria and 
bacterial toxins but also other ‘danger signals’ 
such as cholesterol crystals. Once they have 
formed, they activate the pro-inflammatory 
cytokines interleukin-1 (IL-1B) and IL-18, 
which then call in lymphocytes to battle the 
perceived infection. 

Scientists once believed that the only job of 
the innate immune system was to protect the 
host from external danger. Recently, however, 
a more nuanced understanding has emerged. 
It is now clear that the innate immune system 
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— and inflammasomes in particular — can be 
activated by endogenous molecules known as 
alarmins, released by damaged cells, or danger- 
associated molecular patterns. Cholesterol 
crystals are one example; others are monoso- 
dium urate crystals in gout, cholesterol crystals 
in atherosclerosis, islet amyloid polypeptide, 
which is deposited in the pancreas in type 2 dia- 
betes, and even simple molecules such as ATP, 
which is released from damaged or dying cells. 

The picture is getting murkier. “Finding 
that the innate immune system can actually 
be activated by self-molecules has chal- 
lenged the traditional view that the immune 
system only recognizes non-self as harmful,” 
says Kate Schroder, a molecular cell biolo- 
gist at the University of Queensland in Bris- 
bane, Australia. 


UNDER THE MICROSCOPE 

Researchers are starting to probe the role of 
inflammasomes in providing defence and 
understanding how they distinguish harm- 
ful from harmless. One of the ways they are 
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building up detail is through creating images 
of the complex assemblages using standard 
confocal microscopes combined with fluo- 
rescent tagging. Confocal microscopy uses 
mirrors to focus light on a sample at multiple 
points and depths, building up a three- 
dimensional image. When fluorescent mol- 
ecules are attached to proteins of interest — 
such as those that make up the inflammasome 
— they show up inside the cell samples. 

Such fluorescent imaging is at the heart 
of a study published in August 2013 linking 
Alzheimer’s disease to inflammation-causing 
proteins’. In it, researchers showed that mice 
lacking a molecule necessary for the activation 
of an inflammasome called NLRP3 did not 
develop the disease — suggesting a potential 
target for future treatments. “When you do not 
have inflammasomes, you do not develop any 
Alzheimer’s, which is really cool, says Eicke 
Latz, one of the study’s authors and director 
of the University of Bonn’s Institute for Innate 
Immunity in Germany. “So you could imagine 
that blocking this pathway could be beneficial 
in humans.” 

Latz has also used a combination of laser 
reflection and fluorescence confocal micros- 
copy to identify the way that crystalline materi- 
als interact with immune cells. His team found 
that feeding mice a high-fat diet caused small 
cholesterol crystals to appear in as little as two 
weeks. High levels of these crystals embedded 
in the vascular wall then led to activation of 
IL-1 (ref. 2). The researchers still do not know 
how the crystals activate the inflammasome, 
but they did identify that the complex is the 
trigger for the escalating response. 

The emerging connection between cho- 
lesterol crystals, IL-1 and atherosclerosis — 
which is an inflammatory reaction that takes 
place in fatty blood vessel walls has prompted 
the first clinical trial of a drug to block IL-1. 
In CANTOS, a double-blind, placebo- 
controlled trial run by Swiss company 
Novartis, 17,200 men and women who have 
had heart attacks are being treated with 
canakinumaba human monoclonal antibody 
that neutralizes IL-1. The trial will follow the 
patients for four years and monitor whether 
they have any further cardiac events and other 
health outcomes. 

Another approach looks at inflammasomes 
in macrophages, which are scavenger cells 
that consume dead cells, bacteria and viruses. 
Denise Monack, a microbiologist at Stanford 
University in California, is using confocal 
microscopy along with a technique called array 
tomography to image macrophage inflammas- 
omes responding to bacteria such as Salmo- 
nella or Fransicella, as well as searching for 
genes that influence inflammasome activation. 

In array tomography, Monack’s group cre- 
ates thin serial sections ofa sample of cultured 
cells and stains them with fluorescent antibod- 
ies. The technique was developed by Monack’s 
colleague Steven Smith in 2007, and Monack’s 


lab is the only one to use it to image inflam- 
masomes. Millions of images are snapped 
with a microscope and assembled into a three- 
dimensional structure using software. The cells 
can be stained with special substances packed 
with electrons, allowing researchers to take 
images at higher resolution with an electron 

microscope. 
Imaging macrophages serially — creat- 
ing two-dimensional 


“We can’t image tiles that are 
explain why, but reconstructed com- 
inflammasomes_ putationally into 
seem to be three-dimensional 

playing a role images — increases 
in somany the resolution of the 


sample by an order 
of magnitude (from 
700 nanometres down to about 70). Monack 
says that array tomography combines features 
of modern optical fluorescence and electron 
microscopy with better spatial resolution than 
confocal imaging*, making the technique ideal 
for imaging the molecular architecture of an 
inflammasome. 

Monack has found that not all macrophages 
form inflammasomes, and that when there is an 
inflammasome there is only one per cell. “That's 
one of my burning questions: why is there only 
one inflammasome?” The answer, she surmises, 
may be that there is some kind of anchoring plat- 
form structure in the cell that allows for only one 
complex to form. 


diseases.” 


THE REAL THING 

Dissecting this type of detail, using techniques 
such as array tomography or confocal micros- 
copy, could help identify potential treatments 
for a range of inflammation-related conditions. 
“By examining the [inflammasome] struc- 
ture, we may be able to design small molecule 
inhibitors to block either the formation of the 
enzymatic complex or the processing of the 
cytokines,’ says Ashley Mansell, head of the 
Toll-Like Receptor (TLR) Signalling Labora- 
tory in the Centre for Innate Immunity and 
Infectious Diseases at the Monash Institute 
of Medical Research in Melbourne, Australia. 
One such application might be influenza: 
Mansell is interested in why pandemic strains 
drive hyper-inflammatory responses, whereas 
regular influenza strains do not’. 

Influenza is not the only point of interest — 
researchers are probing other, chronic illnesses 
to find targets for inflammasome inhibition. 
Latz points to preclinical studies that show 
that mice treated with an IL-1 antibody do 
not develop diabetes. His team is using high- 
throughput screening to look for other inflam- 
masome inhibitors, but Latz thinks it will 

take a while to develop 
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— and that hasn't been established,’ he says, 
although he adds that the field is so new that 
more advances will come in the next few years. 

A deeper understanding of inflammasomes 
could also be useful for diagnostics. Mansell is 
working on ways to get an earlier and better 
diagnosis of atherosclerosis or cholesterol prob- 
lems with a blood test, searching for markers of 
inflammasome activation in the serum. 

Mansell says one potential scenario is a doc- 
tor testing a patient’s blood for IL-1, and cor- 
relating that with other risk factors to offer an 
early warning sign. “Inflammasome activation 
is a marker that means there is a disruption of 
homeostasis,” he says. 

Treating or halting inflammasome activa- 
tion early could stop a disease in its tracks. 
Mansell points to hypertension, which is a 
precursor to many illnesses from stroke to 
heart disease. Recent studies indicate a rela- 
tionship between hypertension and inflam- 
masome activation. If this link is proven, and 
a test developed, “we could offer people a way 
to treat or stop the disease from progressing 
further’, he says. 

Pharmaceutical companies are already 
investigating inflammasome-based thera- 
peutics. For example, Idera Pharmaceuti- 
cals, based in Cambridge, Massachusetts, is 
working on treatments for psoriasis, lupus 
and arthritis that block toll-like receptors, 
which mediate the inflammasome response. 
And Navidea Biopharmaceuticals, based in 
Dublin, Ohio, has recently received approval 
for Lymphoseek, a lymphatic mapping agent 
that binds to a receptor known as CD206 on 
the surface of macrophages and dendritic 
cells — both of which are immune cells that 
house inflammasomes. The radiopharma- 
ceutical binding agent allows doctors to see 
lymph nodes in the potential drainage path of 
a tumour. Originally designed for the lymph 
nodes around breast cancer, the agent could 
also point to early inflammasome involvement 
in some 15 diseases, says Fred Cope, Navidea’s 
chief scientific officer. 

Mansell says that this embryonic field 
has extensive options, because inflamma- 
some activation is implicated in such a wide 
swathe of maladies. “Inflammasomes are the 
sparks that set things off — we can’t explain 
why, but they seem to be playing a role in so 
many diseases,” he says. “There have been a 
number of real breakthroughs in this area 
over the past few years.” Bringing inflam- 
masomes further into focus should yield 
many more. # 


Katharine Gammon is a freelance science 
writer based in Santa Monica, California. 
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Imaging tools act as an extra pair of eyes for the surgeon. 


The eyes of 
the operation 


Real-time imaging of a patient’s body is guiding surgeons 
and radiologists past healthy tissue to the diseased cells. 


BY JESSICA WRIGHT 


student, he watched as a team of surgeons 
prepared to treat a child with brain cancer, 
aiming carefully planned radiation treatment 
at a fuzzy X-ray image. 
“I thought to myself, a huge amount of effort 
has gone into designing the treatment for this 
child, and we have all this technology and all 


le 1988, when David Jaffray was a summer 


these people standing around trying to get this 
right, and we can’t see what we're doing,” says 
Jaffray, a medical physicist at the Princess Mar- 
garet Cancer Centre in Toronto, Canada. He has 
worked ever since to engineer solutions to this 
problem. 

Jaffray’s experience shows that in many 
ways, doctors work in the dark. A suspicious 
lump of cancer looks exactly like the rest of 
the breast, for example. The prostate can hide 
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a tumour that can evade multiple biopsies. 
But what the human eye cannot see, precision 
medical imaging can — whether it’s mag- 
netic resonance imaging (MRI) or computed 
tomography (CT), for example. For the past 
twenty years, researchers have worked to turn 
these scans into maps that reveal the hidden 
insides of a patient's body, guiding surgery or 
radiation therapy. 

In 2003, Jaffray helped design the first 
radiation machine with a built-in CT scanner. 
This equipment allows radiologists to trace 
the outline of the tumour before each dose. 
The resulting confidence that they can avoid 
healthy tissue has allowed doctors to increase 
the dose of radiation. In the brain, imaging 
reveals not just the structural outlines of a 
tumour, but which parts can be cut out with- 
out dire consequences. And researchers are 
working to adjust these maps during surgery, 
so they update in real-time, moving and shift- 
ing along with the patient. 


TUMOUR TOPOLOGY 

Image guidance is particularly valuable for 
cancer treatment because it addresses the 
primary challenge: how to remove every 
last bit of a tumour while damaging as little 
healthy tissue as possible. Imaging during 
a breast cancer lumpectomy, for example, 
allows surgeons to remove the small ‘bread- 
crumbs’ of cancer that are often left behind, 
significantly reducing the risk of recurrence, 
says radiologist Ferenc Jolesz, director of the 
National Center for Image-Guided Therapy 
at Brigham and Women’s Hospital in Boston, 
Massachusetts. 

In the early 1990s, Jolesz pioneered the use 
of MRI in operations, taking scans during 
brain surgery for the first time. When this was 
successful, it became clear that the best way to 
guide treatment would be to combine as many 
forms of imaging as possible, says Jolesz. In 
September 2011, a grant from the US National 
Institutes of Health led to the Advanced Multi- 
modality Image Guided Operating (AMIGO) 
suite — a three-room operating suite that 
includes an MRI scanner, a CT and positron 
emission tomography (PET) scanner, and an 
advanced three-dimensional ultrasound and 
navigation system. 

Researchers are exploring how to combine 
the resources at AMIGO to refine treatments. 
Imaging during surgery can address the prob- 
lem of overtreating early-stage tumours, such 
as those found during routine lung CT scans 
on smokers. Small lumps are difficult to locate 
so surgeons may end up removing large pieces 
of lung tissue that will never grow back, says 
Raphael Bueno, a thoracic surgeon at Brigham 

and Women’s Hospital. 
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hook-like device in the lesion. The hook is 
attached to surgical thread that reaches out 
of the lung. During surgery the thread acts as 
a guide, allowing Bueno to snip out only the 
affected tissue. 

But nowhere is accurate targeting more 
essential than in the brain, where the neurons 
that control key functions may snake past a 
tumour or through the target site of epilepsy 
surgery. What's more, a tumour can reorganize 
the brain's function, shifting the neuronal con- 
nections. To address this, surgeons are devel- 
oping ways to glean information from MRI 
scans — such as functional MRI (fMRI) or dif- 
fusion tensor imaging (DTI) — that map not 
only the brain’s structure, but also its function. 

Functional MRI highlights the parts of 
the brain that receive the most blood when 
patients perform a task, revealing which 
regions may be involved in certain functions. 
And DTI analyses the diffusion of water mole- 
cules in the brain that orient in the same direc- 
tion as the long tracts of neurons. Researchers 
can turn these data into maps that paint brain 
tracts different colours according to their 
direction, identifying information highways 
once invisible to surgeons. 

Using these scans to inform surgery has 
dramatically improved patients’ outcomes, 
says Christopher Nimsky, a neurosurgeon at 
the University of Marburg in Germany. “Ten 
or twenty years ago we accepted that we would 
cause 10% or 15% new neurological deficits 
after surgery.’ Thanks largely to the use of 
imaging during surgery, he says, that figure is 
“now down to 2% or 3%, even in the compli- 
cated cases.” 

The functional maps, which doctors have 
begun to use routinely in the past five years, 
help them decide which patients would be too 
impaired by surgery to make it worthwhile. 
But they also give doctors the confidence to 
pursue more aggressive surgery in certain 
cases. Using DTI may reveal that although a 
tumour is in a functional region, it sits further 
away from essential neuronal connections 
than the surgeon originally thought, allow- 
ing them to remove more of the tumour or 
surrounding region. 

These technologies were originally designed 
as research tools. Turning the data into maps 
that report on the location of important brain 
regions is complex and in many ways subjec- 
tive. “The challenge is cutting to the core of 
what information the surgeon really needs,” 
says Alexandra Golby, a neurosurgeon and 
researcher at Harvard Medical School in Boston, 
Massachusetts. “The clinician wants to ask, 
is that tract behind the tumour or running 
[through] the tumour? Which tracts am I likely 
to encounter on my way in? Is this area con- 
nected to that area?” 

Many surgeons now routinely use Slicer, an 
open source software tool introduced in 1999 
that combines structural MRI scans with {MRI 
and DTI data in three dimensions’. Shown all 


at once, the tracts produced by DTI analysis 
look “like a bowl of spaghetti’, says Golby. In 
2011, Golby added a module to Slicer that 
allows surgeons to isolate any of the tracts that 
pass through a boundary they have defined 
around the tumour. Surgeons can also high- 
light a particular spot in the brain, and view 
only those tracts that pass through it. During 
the operation, these maps may be combined 
with the surgical navigation guidance system, 
which uses structural data to show surgeons 
where they are in the brain. 

One crucial feature is that the outlines traced 
on imaging scans accurately reflect the bound- 
aries of the tumour. In fact, different scans, 
such as CT and MRI, may not even match each 
other, says Kristy Brock, a medical physicist at 
the University of Michigan in Ann Arbor. It’s 
fascinating to see how different the tumours 
look on all these things,” she says. 

To get a better understanding of these out- 
lines, Brock is working alongside radiologists, 
oncologists, surgeons and pathologists in a new 
field called correlative pathology. In one study, 
Brock and her colleagues took images of the 
liver before and after it was surgically removed. 
They then sliced it into sections, photographing 
each one to create a three-dimensional picture. 
Finally, they used histology to stain cancer cells 
a different colour from the rest of the tissue so 
they could identify the cancer at the cellular 
level in each slice”. By combining these layers 
of information, researchers can get closer to 
understanding how accurately MRI defines the 
boundaries of a tumour, says Brock. They can 
also compare different imaging technologies to 
determine which scan, or combination of scans, 
most accurately represents reality. 


MAPPING IN REAL TIME 

For even more precision, however, an imaging 
system should take into account the significant 
changes that occur during surgery. Once the 
skull is open, cerebrospinal fluid leaks out 
and the brain bulges from the skull. And as 
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A 3D reconstruction using data from several 
imaging technologies reveals a tumour. 
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the surgeon cuts into the tissue, there are even 
more changes. With an MRI machine in the 
operating theatre, doctors can match the tract 
reconstructions with the shifting reality ofa 
brain in an open skull. 

The challenge is to take the surgical scan as 
quickly as possible and line it up accurately 
with the pre-surgical images, says Gavin 
Winston, a neurologist at University College 
London. Winston is aiming to do this for the 
optic radiation, a vision tract that often passes 
through the site targeted for epilepsy surgery. 
He has developed computer adjustments that 
allow doctors to take a second DTI scan during 
surgery and line this up with the existing DTI 
tracts in a matter of minutes’. 

This innovation can have real benefits for 
the patient’s quality of life. Many patients 
opt for epilepsy sur- 
gery with the hope 


“It’s fascinating of qualifying for a 
to see how driving licence. But 
different the damage to the optic 
tumours look radiation can impair 
on allthese their vision so much 
things.” that they still can’t 


drive, Winston says. 

His team has used 
this technique on 12 patients so far, and all 12 
were able to drive after surgery. 

But taking an MRI or DTI scan during sur- 
gery is extremely difficult, says Golby. “You 
have to stop, get the metal out of the way, and 
then move the patient into the scanner, or 
move the scanner over the patient,’ she says. 
“We've come to take it for granted that we can 
get these gorgeous pictures of the insides of 
people’s bodies, but it takes a huge technical 
tour de force to do so.” Golby is developing 
simpler methods to keep surgeons updated. 
One option is to use ultrasound, which is easy 
to use and cheap. Ultrasound provides much 
sparser data than MRI, she says, but may be 
sufficient to tell the surgeon when something 
significant has changed in the brain. 

All these approaches allow researchers to 
consider each patient’s biology and to base 
treatment on their unique response. “The idea 
that treatments can be more precise by integrat- 
ing imaging is sort of obvious,’ says Jaffray. “But 
the technological advance that allows us to do 
that in a non-invasive way is pretty remark- 
able,” he adds, considering how much the 
tumour changes shape over time. He envisages 
“fleets of multimodal treatment machines” that 
can make these adjustments consistently — 
systems that will, in effect, bring superhuman 
vision to those seeking to repair the human 
body. = 


Jessica Wright is a freelance science writer 
based in New York City. 
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Photoacoustic microscopy reveals the microvasculature of a mouse brain, while ultrasonic microscopy gives a glimpse of the skull (grey). 


TECHNOLOGY 


Multiple exposure 


Combining imaging techniques can provide a wealth of information about disease. 


BY NEIL SAVAGE 


at the University of Geneva in Switzer- 
land, built a low-cost positron emission 
tomography (PET) scanner. The design left 
some spaces in the instrument's structure, and 
Townsend wondered whether he could fill 
them, and improve the machine, by squeezing 
a second scanning technology into the gaps. 
A doctor friend told him that surgeons were 
more familiar with the anatomical information 
provided by computed tomography (CT), so he 
added that, and the PET-CT scanner was born. 
At first, many in the medical establishment 
were sceptical about the instrument's potential. 
“They all thought David Townsend was kind 
of loony,’ says Michael Vannier, a radiologist at 
the University of Chicago. “There was a lot of 
foot dragging associated with PET-CT” 
But what was unfamiliar 15 years ago has since 
become the norm. These days, “any sensible 
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person would not use PET alone,” says Vannier, 
adding that the PET-CT scanner has “vastly 
improved doctors ability to identify the stage of 
alymphoma, allowing more nuanced treatment 
decisions: Simon Cherry, a biomedical engineer 
at the University of California, Davis, agrees. “It’s 
almost impossible to buy a PET scanner without 
aCT scanner attached to it now,’ he says. 

That success has spurred other efforts to 
combine imaging modalities. None of the 
many ways of looking inside the human body 
is perfect, but merging the strengths of two 
or more technologies may allow physicians 
to see details they have never seen before and 
improve the detection, diagnosis and treat- 
ment of ailments ranging from cancer to heart 
disease or Parkinson's disease. 

Imaging techniques mainly show either 
structural information — the physical shape 
of an organ or a tumour — or functional 
information, such as which molecules are pre- 
sent or what metabolic activity is occurring. 
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But no single technique is optimal for both. 
Coronary angiography, for example, in which 
X-rays image a contrast agent flowing through 
the heart’s blood vessels, provides “exqui- 
site detail” of those vessels, but doesn’t show 
whether the cells in the heart muscle are living 
or dead, says Vannier. If they’re dead, coro- 
nary bypass surgery would be ineffective. “You 
could improve the circulation,” Vannier says, 
“but the heart wouldn't beat any better.” 
Similarly, X-ray tomography can spot tiny 
breast tumours but has trouble telling which 
are benign and which are malignant. This lack 
of functional information leads to a lot of false 
positives, followed by unnecessary surgery, says 
Lihong Wang, a biomedical engineer at Wash- 
ington University in St Louis. “If you can tell the 
function, you can get more information,” Wang 
explains. “The end result is higher accuracy.” 
Wang is trying to improve endoscopy by 
combining conventional ultrasound with a 
technology called photoacoustics. Ultrasound 
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endoscopy provides high-resolution images 
of structures and is widely used to look for 
oesophageal or colorectal cancer, for example. 
But its contrast is low, so it does not readily dis- 
tinguish between blood vessels and lymphatic 
vessels, or healthy and diseased soft tissue. 

Photoacoustic tomography fills that gap 
by helping to identify different types of mol- 
ecules’. To obtain a photoacoustic image, 
Wang delivers pulses of light through an opti- 
cal fibre to the tissue being examined. When 
the tissue absorbs the light, it heats slightly 
and expands, sending outa pressure wave that 
can be detected by the same sort of receivers 
that pick up the ultrasound signal. Different 
substances in the body — water, lipids, mela- 
nin — absorb different wavelengths of light, 
so by selecting the wavelengths, investigators 
can identify what they're looking at. Two par- 
ticular wavelengths of visible light, 562 nm and 
584 nm, can distinguish between haemoglobin 
that’s saturated with oxygen and haemoglobin 
that lacks oxygen. This can reveal a tissue’s rate 
of oxygen consumption, which is a measure 
of metabolic function. Near-infrared light 
with a wavelength of about 1,200 nm can pick 
out lipids and allow doctors to tell if a plaque 
inside an artery is vulnerable to breaking off 
and causing a clot. Wang suggests that the 
absorption of photons by DNA could allow 
photoacoustic imaging of cell nuclei, making 
it possible to perform a sort of on-site biopsy 
without removing any tissue. 

Both photoacoustic and ultrasound imaging 
systems use an ultrasonic transducer to process 
the signals, so they can easily be incorporated 
in the same machine, Wang says. He is cur- 
rently working with Philips Healthcare to build 
a commercial version. 

The structural information provided by ultra- 
sound can also be combined with the molecu- 
lar information available through confocal 
microscopy, an optical technique that provides 
high-resolution images, allowing scientists 
to identify molecules when a fluorescent tag 
latches onto them. “They’re perfectly comple- 
mentary techniques,’ says Christopher Contag, 
a microbiologist at Stanford University’s Molec- 
ular Biophotonics and Imaging Laboratory. He 
suggests going even further in combining tech- 
niques: adding a third, such as photoacoustics, 
would make the images even richer. “Every 
modality gives youa little bit different informa- 
tion, so a combination of them gives you some 
varieties to choose from, he says. 

Contag’s approach could provide enough 
information for doctors to perform histopa- 
thology inside a living patient, instead of ona 
piece of excised tissue in a lab. This would not 
only reduce the need for biopsies, but could 
also improve the accuracy of the diagnosis. 
Tissue that has been removed and stuck to a 
microscope slide might be dried out, reshaped 
or otherwise changed by the process. Leaving 
it in place avoids those problems. 

Combining modalities in this way could also 


extend the use of molecular probes. Using dyes 
already approved by the US Food and Drug 
Administration to stain tissue inside a patient 
could make Contag’s vision of in vivo biopsy 
possible — and new molecular probes might 
make it possible to detect cancers at an ear- 
lier stage. One probe, developed by Stanford's 
Matthew Bogyo, tests for legumain, a pro- 
tease produced by most cancers in their early 
stages. Such a probe would make it easy to spot 
the early signs of cancers in the microscope, 
Contag says. “If there’s a tumour, it’s activated 
and it lights up like a beacon” 


COMPARING CONTRASTS 

Another approach to making images richer is 
to mix together not data from different tech- 
nologies, but different results from the same 
technology. Multispectral magnetic resonance 
imaging (MRI) isa case in point. Conventional 
MRI uses radio-frequency pulses and changes 
in the magnetic field to create an image. Each 
specific sequence of pulses provides its own 
type of image, or contrast, making different 
types of tissue easier to see. 

A group led by neuroscientist Suzanne Cor- 
kin of the Massachusetts Institute of Technol- 
ogy combined four of these contrasts to look 
for early signs of the brain damage caused by 
Parkinson’s disease’. They wanted to measure 
the volume of a brain region called the sub- 
stantia nigra, which is known to deteriorate in 
Parkinson's. Post-mortem examinations of the 
brains of Parkinson’s 
patients have led to 
a theory of how the 
damage progresses, 
but doctors had 
been unable to see 
the changes in a living brain. 

Two of the contrasts provided by differ- 
ent pulse sequences show the boundaries of 
the substantia nigra, but individually each is 
a bit fuzzy, says David Ziegler, a neurologist 
at the University of California, San Fran- 
cisco, who worked with Corkin on the MIT 
project. A third contrast shows white matter 
appearing very bright, but cannot distinguish 
it from cerebrospinal fluid, whereas a fourth 
makes the fluid look black, so putting those 
two contrasts together makes it clear which 
is which. The team took images of volunteers 
using the four contrast methods in sequence 
before combining them into a single image. 
“Now the boundaries of the substantia nigra 
and the red nucleus [another brain structure] 
just sort of pop,’ Ziegler says. Although such 
a small study (29 Parkinson's patients and 27 
controls) could not provide definitive results, 
the images seemed to confirm what the post- 
mortem data had suggested about the way the 
disease progressed. 

Multispectral MRI could lead to the earlier 
detection of Parkinson's, and perhaps earlier 
treatment, and might provide a greater under- 
standing of the mechanisms of the disease. “We 
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can start going deeper and deeper to detect the 
earliest losses that might be associated with 
Parkinsons,’ Ziegler says. They might learn 
even more by marrying that ability to PET 
scans or electroencephalograms. 

The trick to multispectral MRI is to take 
all the scans in the same session. Any MRI 
image has some distortions, which can be cor- 
rected for. But with multiple scans, the images 
must be aligned correctly for that to work; 
scans from separate sessions would be hard 
to line up. “You can’t take differently distorted 
images and combine them in any meaningful 
way, Ziegler says. The MIT team scanned the 
patients with the four contrast methods back to 
back, then immediately repeated the sequence, 
ina process lasting about an hour. They then 
averaged the two sets to correct for artefacts 
from movement, and then combined the four 
contrasts into one image. 


CLEAR GUIDANCE 

The latest combined technology is the PET- 
MRI scanner, designed by Cherry and only just 
coming onto the market. Many major US hos- 
pitals don’t yet own one. Using PET, which is 
useful for finding biomarkers, adds functional 
information to the MRI image. The combina- 
tion could improve imaging of the brain and 
of the soft tissue in the pelvic area, helping to 
identify such diseases as bladder or brain can- 
cer. Cherry says the combination provides the 
best of both worlds: “PET is the most sensi- 
tive technology out there. MRI is the highest- 
contrast technology.” 

At roughly US$5 million per machine 
(roughly double the cost of high-end MRI), 
PET-MRI faces a high hurdle to widespread 
adoption. And some scientists are sceptical. 
“We don't yet have examples where PET-MRI 
is clearly superior to PET-CT,’ Cherry admits. 

But Vannier is afraid that being too cautious 
in developing new combinations of imaging 
techniques could hobble medical progress. As 
medicine gets better at detecting more cancers 
and other abnormalities, he says, it’s important 
to distinguish those that are potentially lethal 
from those that are unlikely to become life 
threatening. And better identification of vulner- 
able plaques could revolutionize coronary care 
and minimize unnecessary treatments that can 
do more harm than good. “The ultimate goal is 
to not over-treat or do one size fits all,” he says. 

Imaging that combines structural and func- 
tional information might not only increase 
early detection, but also guide doctors about 
what to treat and what to leave alone. “The 
imaging,’ Vannier says, “will provide the tool 
that in clinical practice will allow you to confi- 
dently make the decision” = 


Neil Savage is a science and technology writer 
based in Lowell, Massachusetts. 
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Will machines be able to judge a patient’s prognosis? This prototype microscope aims to do part of the job. 


The computer 
will see you now 


From image-analysis software to lens -free microscopes 
that fit on a mobile phone, new tools are providing 
pathologists with clearer and more informative images. 
BY KATHERINE BOURZAC 
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rians such as Galileo, Antonie van Leeu- 

wenhoek and Robert Hooke learned to 
grind lenses and make the first microscopes, 
revealing the hidden landscapes of life. They 
saw for the first time the cells in cork, blood 
and other tissues, and van Leeuwenhoek found 
swimming ‘animalcules’ in dental plaque and 
observed the movement of sperm. 

Physicists and engineers are now trying to 
bring about a similar shift in perspective for 
microscopy. In most pathology labs, doc- 
tors diagnose diseases by poring over tissue 
slices on glass microscope slides — classifying 
tumours, for example, based on subtle visual 
cues that are difficult to quantify. But this is 
starting to change. Just as lenses once revealed 
vistas that were previously invisible to the 
human eye, so software is opening up a new 
window on biology. 

The latest digital tools make it possible to 
do a visual search in microscopy images, auto- 
mate diagnosis, and sync image data with the 
genomic profiles of tumours. Some researchers 
are even doing away with lenses altogether, 
creating computational microscopes based 
on inexpensive hardware that could be used 
for point-of-care diagnostics, particularly in 
poor areas with few doctors. 
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BIG DATA 

Pathology has remained stubbornly analogue 
and qualitative, however. The experienced 
pathologist’s main tools are glass slides, a com- 
pound microscope whose design has hardly 
changed in more than 200 years, and eyes that 
have seen thousands of tumours. “Most of a 
pathologist’s medical decisions are based on 
morphology,’ the structural details of cells 
and tissues revealed under a microscope, says 
David Rimm, a pathologist at the Yale School 
of Medicine in Connecticut. 

Just because a method is old is no reason to 
abandon it, of course. But advocates of digital 
pathology worry about inconsistencies that 
can lead to false negatives and misdiagno- 
ses. Experienced pathologists are better than 
younger ones at identifying rare tumours, but 
they often disagree with one another and even 
with their own assessment of the same sample 
from weeks before. 

One hurdle to digitizing clinical microscopy 
is the size and complexity of the images, says 
Metin Gurcan, who specializes in biomedical 
informatics at Ohio State University and was 
an early advocate of digital pathology. First, 
a biopsy is sliced into sections and placed on 
multiple slides. A digital image of a single slide, 
magnified under the microscope, has about 10 
billion pixels and requires about 30 gigabytes 
of memory. A typical prostate biopsy, for 
example, uses more than 20 slides and needs 
about 600 gigabytes. 

That's a lot of information for pathologists to 
scan through — and a lot of data for software 
to sift. “The number and type of cells found 
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in these images is mind-boggling,’ Gurcan 
says. One way to deal with this complexity is 
to use software that learns to recognize things 
in images the same way people do, but faster 
and more consistently. 


VISUAL LEARNERS 

Just as people learn by seeing many examples, 
so can software. In 2011, Harvard Medical 
School pathologist Andrew Beck built a tool 
called C-Path (for Computational Pathologist) 
by feeding learning software with images of 
breast-cancer biopsies from 248 patients, along 
with survival data’. The software learned to 
grade the severity of breast cancer and predict 
patient survival. 

A human pathologist who looks at these 
biopsies under the microscope relies primar- 
ily on three features specific to cancer cells to 
decide how aggressive the tumour is. Do the 
cell nuclei have an unusual shape? Are the 
cells dividing? And are the cells connecting 
with one another as normal, or are they iso- 
lated? Pathologists qualitatively score each of 
these features to determine the tumour grade, 
a description of how aggressive the tumour is. 

The C-Path system works by segmenting 
images into small regions called ‘superpixels: 
It identifies cell nuclei and cytoplasm within 
each superpixel, and compares the qualities of 
each superpixel — such as colour, texture, size 
and shape — with those of its neighbours. For 
breast cancer, this comparative analysis gener- 
ates features related to both a sample's global 
structure and its fine-scale details, such as the 
average distance between the nuclei of cancer 
cells and normal cells. 

After crunching the training set of images, 
C-Path came up with 6,642 features, describing 
not only the tumour cells themselves, which 
human pathologists focus on, but also the sur- 
rounding connective tissue, called the stroma. 
Indeed, Beck found that the morphology of 
the stroma was a better predictor of survival 
than that of the cancer cells alone: an area of 
stroma that was uniform was associated with 
a good prognosis, whereas stroma that was 
infiltrated by epithelial cells indicated more 
aggressive cancer. Based on its analysis of thou- 
sands of features, C-Path was able to predict 
patient survival more accurately than standard 
pathological analysis. Beck is now training the 
software on a broader range of samples, includ- 
ing images of whole slides, and normal breast 
tissue samples. 

It is possible that highly experienced 
pathologists also look for some of the thou- 
sands of features spotted by C-Path but just 
can’t describe them in words. Rimm com- 
pares the experience of spotting a tumour 
with recognizing your uncle in a photo. You 
can’t articulate exactly how you know he’s 
your uncle — is it his nose, eyes, clothing? 
You just know it’s him. But the computer can 
quantify features in an image, and the analysis 
is repeatable. 


Richard Levenson, a pathologist at the Uni- 
versity of California, Davis, says that software 
such as C-Path has the potential to replace 
pathologists in assigning grades to tumours. 
Others believe that the right place for software 
is as an aide to help physicians navigate large 
digital images in real time — a second set of 
very sharp, superhuman eyes. Ulysses Balis, a 
specialist in pathology informatics at the Uni- 
versity of Michigan in Ann Arbor, is develop- 
ing this latter kind of tool: an all-purpose visual 
search program called SVIQ. 

Balis demonstrates SVIQ with a digital 
image of a slice of colon adenocarcinoma. He 
can bring up different fields of the image and 
zoom in and out. If he finds something inter- 
esting — such asa cell that appears to be divid- 
ing — and wants to see if there are other similar 
features in the image, he clicks a ‘scar’ button 
and the software highlights all the parts of the 
image that look similar’. In this case, all the 
dividing cells turn red. The concept is similar 
to smartphone apps such as Google Goggles or 


Raw data of a x2 objective lens (top) before being 
rebuilt as a high-resolution image (bottom). 


TinEye, which allowa user to upload an image 
to find out what it is and where you might 
find more like it. Balis demonstrates the sys- 
tem from his laptop; the image analysis takes 
about 10 seconds. “This is the simplest possible 
search structure for a two-dimensional image,” 
he says. “It’s a good quick-and-dirty tool.” 
The SVIQ software can help pathologists 
find sites of interest in the visually over- 
whelming landscape ofa digital slide. Whereas 
C-Path is designed to make diagnostic deci- 
sions, SVIQ is a way to make digital pathology 
images more user friendly — and to extract 
information from them quickly. Jason Hipp, 
who works on pathology informatics at the 
National Cancer Institute in Bethesda, Mary- 
land, uses SVIQ to quantify features over an 
entire slide. Instead of scoring the number of 
cells that are dividing in a few visual fields, 
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as pathologists do, he says, SVIQ can count 
every single dividing cell in the entire tissue 
slice. “When we have to count and measure, it’s 
time consuming,” he says. Using SVIQ could 
speed up the pathologist’s work and provide 
more morphological data. 

One of Hipp’s research goals is to integrate 
SVIQ into the process of screening patients for 
cancer clinical trials. This often starts with a 
genetic test to find eligible patients. However, 
for tumours that are dispersed, it can be diffi- 
cult to find enough cancerous tissue to perform 
the genetic screen, as the cancer’s genetic sig- 
nal can be lost in the noise from normal cells. 
It’s also difficult to do genetic tests on samples 
taken by needle biopsies, which are less inva- 
sive but produce less tissue to work with, says 
Hipp. In these cases, pathologists have about 25 
minutes after taking a biopsy from the freezer 
to identify and hand-dye cancerous portions 
of a tumour slice before it deteriorates. A laser 
is used to remove the undyed sections, leaving 
enriched cancer cells for genetic screening. 

The SVIQ software can help to digitize this 
entire process. A pathologist finds an area of 
the image where there are cancer cells, uses 
SVIQ to highlight the rest of the cancer, and 
this map is then sent to the laser cutter. The 
process takes just 5 minutes. 

At the Institute of Cancer Research in Lon- 
don, bioinformatics researcher Yinyin Yuan 
aims to map all the cell types in a tumour 
alongside their gene expression data. “The 
different cell populations in a tumour create a 
complex landscape that is an obstacle to accu- 
rate diagnosis,’ she says. A sequencing study 
that samples part of a tumour cannot capture 
the full picture: it blurs the role played by sup- 
port cells and misses the heterogeneity of the 
cancer-cell population. These issues affect 
patients’ prognoses and how they will respond 
to different kinds of therapy. 

In 2012, Yuan developed software to clas- 
sify the identity and distribution of each of the 
million or so cells from 300 whole-tumour 
slides of breast-cancer biopsies, and then 
integrated this with other ‘“-omics’ data. A 
human pathologist would take too long to go 
through so many cells in this manner, but a 
cluster of 100 computing cores, each with the 
power of a PC, can do the job overnight. The 
output from Yuan's software’ is not an image, 
but data, analysed and collated with other data 
about the tumour. Yuan found that patients 
with immune cells that infiltrated the tumour 
had a better prognosis, and this prediction was 
strengthened when the image data were cou- 
pled with gene expression data. This result is 
not obvious to the eye of a pathologist staring 
at a slide and glancing at a list of gene expres- 
sion data, but it becomes clear when Yuan’s 
software analyses the image. Yuan is now 
expanding the project to study ovarian and 
lung cancer. 

“What's happening in medical imaging is 
similar to what happened in astrophysics,” 
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says Yuan. When astronomers got access to 
powerful telescopes and digital images, they 
didn't insist on counting every star — they 
let computers take over such tedious tasks. It 
should be the same with digital microscopes 
and cells, she says. 


MATHEMATICAL LENSES 

Using software to analyse digital images will 
make the pathologists’ job easier. But it will 
also lead to new kinds of hardware. Today, 
creating digital microscopy images means 
scanning microscopy slides, and this is slow 
and expensive. Researchers are now build- 
ing microscopes that can do both jobs, rely- 
ing more on the power of software and less on 
lenses and other expensive hardware. 

Slide scanners take multiple images, 
mechanically repositioning the slide under 
a microscope each time and then stitching 
the images together. “Mechanical scanning 
is slow,’ says Changhuei Yang, who develops 
microscope technologies at the California 
Institute of Technology in Pasadena. Because 
of their expense, these scanners are not typi- 
cally found in community hospitals. Yang’s 
solution is to increase the field of view and 
resolution of conventional microscopes. In 
July 2013, Yang hacked a low-resolution light 
microscope to create a high-resolution micro- 
scope with a wide field of view that can create 
whole-slide images with cheap hardware’. In 
conventional microscopes, low-power lenses 
provide a wide field of view at the expense 
of resolution; high resolution only occurs 
in small fields of view, hence the need for 
mechanical scanning. Yang’s computational 
microscope can image an area as large as 
120 square millimetres at a resolution of 0.8 
micrometres; a comparable standard micro- 
scope offers a field of only 1.1 square millime- 
tres at this resolution. 

The trick is to use an array of light-emitting 
diodes programmed to sequentially illuminate 
the sample with three different colours of light 
from several different angles; the microscope 
records a picture each time. These images 
are then combined, picking apart the way 
the sample bent or changed the colour of the 


different light sources, to reconstruct a single, 
large-area, high-resolution image. Beck, who 
invented C-Path, says these images are compa- 
rable in quality to those made with expensive 
slide scanners. It’s also more efficient, Yang 
says: “The lead time is shorter, so the num- 
ber of samples a pathologist can examine can 
increase.’ Yang has just started up a company, 
Clearbridge BioPhotonics, based in Singapore, 
to commercialize the technology. 

Some scientists are going one step further 
and building totally lens-free microscopes. 
Aydogan Ozcan, an electrical engineer at the 
University of California, Los Angeles, is devel- 
oping microscopes that are basically just light- 
sensing electronic chips of the kind found in 
consumer electronics, but altered to cope 
with wet biological samples. The fancy part is 
Ozcan’s software, which does the same thing 
as a physical lens: it transforms blurry inter- 
ference patterns into focused images of cells. 
His compact microscopes’ reveal the same 
details as those with lenses — those made 
with state-of-the-art chips have a resolution 
of hundreds of nanometres, clear enough to 
reveal the nuclei of cells. Sample preparation 
is similar to that for conventional microscopy. 

The components needed to build these 
microscopes cost just a few dollars, and the 
calculations can be performed by the proces- 
sors found in mobile phones. In fact, for dem- 
onstration purposes, Ozcan has built several 
microscopes attached to mobile phones. “We 
want to empower point-of-care offices or small 
clinics to work like a hospital lab,” says Ozcan. 

He is also experimenting with crowd- 
sourcing diagnostics. He uploaded the images 
of blood cells made by his microscopes to 
an online game (go.nature.com/mnmsmy) 
that teaches non-experts to recognize cells 
infected with malaria. The same images are 
shown to many different players, and by 
statistically combining the answers — after 
removing those clearly trying to upset the 
system — Ozcan’s software generates the 
same diagnosis as professional pathologists 
99% of the time’. The idea isn’t to ‘gamify’ 
pathology — although the games might 
serve as training tools for medical students 
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and lab technicians. Eventually, Ozcan says, 
smart software will be able to take over from 
human pathologists. In the meantime, says 
Ozcan, “I think we'll see hybrid modalities 
like this before machine learning takes over 
completely.” 

Innovations in computing are set to trans- 
form the field of pathology, says Alan Nelson, 
a physicist and chief executive of VisionGate, 
a company based in Phoenix, Arizona, that is 
developing three-dimensional imaging for the 
automated detection of cancer cells in sputum 
and blood. “A machine doesn’t give an opinion 
— it can produce data and absolute diagnosis 
based on statistics,’ he says. The system could 
increase screening rates and help patients get 
the right treatment sooner. 

Nelson previously was the lead inventor of 
the only automated cancer screening test cur- 
rently on the market. His cervical cancer test, 
developed at his company NeoPath, received 
approval from the US Food and Drug Admin- 
istration in 1996 and is now marketed by 
Becton Dickinson. This test uses a processor 
that is custom built for the specific problem 
of spotting cancer cells in pap smears. The 
machine is loaded with hundreds of slides that 
are scanned automatically all day. 

Today’s computers are capable of much 
more. Nelson says that microscopes aided by 
software are now showing biologists and doc- 
tors things they've never seen before. Ozcan’s 
lens-free microscopes have revealed new pat- 
terns of helical motion in sperm, and Vision- 
Gate's three-dimensional images can show 
pathologists hundreds of previously unseen 
features. “We can see the texture of the inside 
of the nuclear surface ofa lung cancer cell, and 
measure the length of the short arm of chromo- 
some six,’ Nelson says. “My god, it’s beautiful!”m 


Katherine Bourzac is a freelance science 
writer based in San Francisco, California. 
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PERSPECTIVE 


hether it is Walmart’s one million customer transactions 
W's: hour, the 220 billion photos on Facebook, or decod- 

ing the human genome’ 3 billion base pairs, dealing with 
vast quantities of data provides an opportunity to extract and exploit 
previously hidden information. Such is the impact of ‘big data that 
the World Economic Forum considers it to be a new class of economic 
asset, comparable to commodities such as gold. The rush to mine this 
new resource is now underway. 

Medical imaging data, which are accumulating in clinics in ever 
larger amounts, seem to be an ideal target for the ‘big data approach. 
This information could potentially provide significant insights into 
health and disease. So far, however, big data — or perhaps more 
appropriately, the ‘big picture’ — has not had much impact on this 
information-rich environment. 


WHAT A WASTE 

Clinical images tend to be used just once, 
by a single clinician for a single patient, 
before being left to gather dust. This rep- 
resents a squandering of resources. Work- 
ing cooperatively, whether at the local, 
regional, national or even international 
level, researchers and clinicians could 
use these images to identify trends and 
correlations, bringing scientific and clini- 
cal benefits. But first we need to under- 
stand how to harness the current clinical 
imaging workflow to capture these data. 

The move towards using the big pic- 
ture offers two immediate benefits. 
First, the image content is self-selecting: 
patients are being referred for investiga- 
tion of medical problems, so their images will reflect conditions 
of clinical relevance and importance. Second, the image data are 
free of charge, or have already been purchased through current 
healthcare systems. We could therefore assemble an inexpensive 
foundation for population-based studies that would otherwise be 
financially unsupportable. 

Consider, for example, the looming medical crisis of dementia. 
Ideally, the focus will be on identifying at-risk individuals so that 
prevention and early treatment measures can be deployed. Imaging 
could hold the key. But in the early stages of dementia the visual 
clues are likely to be subtle, and identifying them requires inves- 
tigation of the small subset of the population with the potential to 
progress. Capturing a sufficient number of appropriate individuals 
for study would mean casting the imaging net very wide, at immense 
cost. However, patients with symptoms potentially indicating early 
dementia — such as vague forgetfulness — may already have been 
imaged in the course of their clinical visits. What if all the imaging 
data could be collated? The power of the combined data would allow 
the small signal contained within the images, denoting early sub- 
clinical disease, to rise above the background noise. 

Research, as opposed to clinical, image networks are already being 


The big picture 


Many medical images are used once then filed away. This trove of clinical 
data should be made available to biomedical researchers, says Alan Moody. 


WE COULD HAVE A POTENT 
NEW TOOL TO ACCELERATE 
NOT ONLY BIOMARKER 
DISCOVERY BUT ALSO 
THERAPY DEVELOPMENT. 


built. Examples include the Alzheimer’s Disease Neuroimaging Ini- 
tiative and the Canadian Atherosclerosis Imaging Network. How- 
ever, these networks are pre-defined by the disease under study; the 
patients recruited already have overt disease. There remains a need 
to explore early disease, or pre-disease, at a population level. This 
can best be achieved by building clinical image networks, allowing 
recruitment at a far larger scale — a job that will require the re- 
purposing of existing clinical imaging data to create the big picture. 
To advance towards this goal, we must change our view of image 
data. Currently, the development of imaging techniques, image acquisi- 
tion and analysis, and qualitative interpretation of the image, is done by 
experts whose aim has been to make their part of the data chain as good 
as possible — in particular, to produce high-quality images and improve 
diagnosis. The next, crucial step is to create 
population-wide image data repositories 

that are available to researchers. 


MAKING IT WORK 

To exploit this resource, two things are 
needed. The first is a new breed of image- 
data scientist. These specialists will be 
data prospectors searching for a signal 
arising from the data, viewing combined 
data rather than individual studies. 

The second is a user-friendly network 
in which to go prospecting. Helpfully, 
clinical images are now commonly digi- 
tized and stored on a picture archiving 
and communication system, where they 
can be examined and categorized using a 
variety of analytical techniques. The bio- 
medical researchers who will use these 
systems must be closely involved in the design of the processes for 
storing, managing and accessing the data. 

It will be important to embed structured reporting within the big 
picture. Clinical images from cancer patients, for instance, generate 
information relating to the primary tumour, lymph-node involve- 
ment and metastases. Combining data ona large scale will accelerate 
insight into primary tumour growth characteristics and associated 
disease spread. 

Ethical standards dictate that the use of such clinical data in 
a research setting requires consent from each patient, or all- 
encompassing institutional permission for the pooling of anonymized 
patient data. Addressing these and other similar requirements will 
be challenging, but if successful, we could have a potent new tool to 
accelerate not only biomarker discovery but also therapy develop- 
ment. Institutions that accept these challenges will be seen as pio- 
neers, exploring this new natural resource and potentially reaping 
the rewards from the wealth of data it contains. m 


Alan Moody is chair of the Department of Medical Imaging at the 
University of Toronto, Canada. 
email: alan.moody@sunnybrook.ca 
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NEXT-GENERATION SCANS 


Proton computed tomography records the position, direction and energy loss from a proton beam as it traverses a patient’s body. 


Seeing into the future 


From magnetically tagged sugar to smoke-sensing surgical knives and beams of high- 
energy protons, the next wave of imaging technologies will provide a clearer view of the body. 


BY PETER GWYNNE 


edical imaging has advanced rapidly 
M: the early years of the twenty-first 

century. Doctors can now observe 
events at the molecular level, examine the 
characteristics of individual heartbeats, and 
study processes in the brain in minute detail — 
tasks all but impossible a decade ago. “We are 
entering the age of precision medicine,’ says 
Roderic Pettigrew, director of the US National 
Institute of Biomedical Imaging and Bioengi- 
neering in Bethesda, Maryland. “We try to be 
precise in diagnosing, fashioning treatments, 
targeting treatments, delivering treatments, 
and monitoring the effects of treatments.” 

Much of the progress has stemmed from 
improvements in existing technologies, such 
as computed tomography (CT), ultrasound 
and magnetic resonance imaging (MRI). “MRI 
now allows you to track the diffusion of water 
molecules in the brain with such precision that 
you can compute their trajectories along fibre 
pathways,” Pettigrew says. 

The latest imaging methods emerging from 
the laboratory promise to complement these 
advances. They can detect and monitor can- 
cers, for example, locate individual cells for the 
delivery of drugs, and provide unprecedented 
accuracy for surgically treating heart disease 
and other conditions. Several technologies 
have yet to reach the preclinical stage, but 


others have begun the journey to clinical tri- 
als. In most cases, they perform tasks currently 
carried out by conventional systems, but do so 
faster, more precisely and more safely. 


A SPOONFUL OF SUGAR 
One such advance in safety relates to a way 
of identifying tumours. Tumours are avid 
consumers of glucose, so patients are usu- 
ally given radioactively labelled analogues 
of glucose, which congregate in the tumours 
and are detected by positron emission tomog- 
raphy (PET). But the danger of radioactive 
exposure prevents this method being used in 
certain individuals, such as young children 
and pregnant women, and limits the number 
of doses for other patients. A team headed by 
Simon Walker-Samuel at University College 
London has developed a method that avoids 
this problem by labelling glucose magnetically 
with bursts of radio waves instead, so it can be 
detected by standard MRI. This non-invasive 
approach is safer — patients merely need to 
take a sugary drink, rather than a radioactive 
isotope. It also enables medical teams to differ- 
entiate among various types of tumour ‘ allow- 
ing them both to determine the appropriate 
therapy more effectively and to assess its effect. 
The technique — dubbed glucoCEST, for 
glucose chemical exchange saturation transfer 
— measures the exchange of protons between 
the hydroxyl groups in glucose molecules and 
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water molecules in biological tissue. The pulses 
of radio waves alter the magnetic character of 
the protons in the hydroxyl groups, masking 
the signal from water molecules detected by 
MRI. “The effect is small, but can be measured 
if we repeat the experiment a large number of 
times,” says team member Xavier Golay. 

The researchers applied the technique to 
two types of human colorectal tumour trans- 
planted into mice. Studying MRI images taken 
before they injected glucose into the tumours 
and one hour after injection clearly revealed 
differences between the tumour types, as dif- 
ferent tumours consistently take up different 
amounts of glucose. The researchers are now 
starting human studies: they are recruiting 
patients with tumours in their neck, and have 
already scanned about a dozen. For precision, 
the team injected the glucose into the mouse 
tumours, but the human patients receive it in 
the form ofa drink. In future, the technology 
may not be limited to tumours. “One could 
imagine using it for assessing any organ with a 
high glucose consumption — for example, the 
heart or the brain,” Golay says. 

Other researchers are expressing cautious 
optimism about the technique. “We feel it is 
feasible but will require some more MRI devel- 
opment, because the human studies will have 
to be done at magnetic fields much lower than 
for the animals,’ says radiologist Peter van Zijl 
of Johns Hopkins University in Baltimore, 


Maryland, whose team is performing its own 
human studies of glucoCEST technology. 

Another technique, magnetic particle imag- 
ing, which was developed in 2001 by scientists 
at Philips Research in Hamburg, Germany, can 
potentially provide a faster, more sensitive and 
safer alternative to the angiography procedures 
used to assess heart disease, particularly during 
operations that require simultaneous imaging 
(see “The eyes of the operation; page S88). The 
technique uses magnetic tracers, rather than 
the chemical contrast agents normally used in 
angiography — an important benefit because 
some patients, such as those with chronic kid- 
ney disease, cannot safely excrete the standard 
angiography tracers, iodine and gadolinium. 

Magnetic particle imaging relies instead 
on iron oxide nanoparticles that are injected 
into the bloodstream. The nanoparticles are 
superparamagnetic, which means they have 
an average magnetism of zero but can be mag- 
netized by an external magnetic field — even 
the weak field generated by a scanner. The pro- 
cess causes the particles to emit small electro- 
magnetic signals that the scanner can detect. 
Changes in the concentration of the nanopar- 
ticles as they sweep through the bloodstream 
make it possible to monitor such critical fac- 
tors as blood supply to the heart, the speed of 
blood flow in the heart, and other data critical 
to coronary surgery. 

So far, researchers have built prototype scan- 
ners only suited to small animals, but Steven 
Conolly, who works on magnetic imaging at 
the University of California, Berkeley, says the 
technology has the potential to revolutionize 
biomedical imaging. Anna Samia, a nanomate- 
rials scientist at Case Western Reserve Univer- 
sity in Cleveland, Ohio, says the technology's 
contrast and sensitivity will exceed those of 
imaging methods such as MRI, X-rays, ultra- 
sound, PET and CT scans. She adds that it 
could ultimately be used for the in vivo tracking 
of stem cells and the imaging of inflammation. 


A SMOKING GUN 

Working literally at the cutting edge of surgery, 
a UK-Hungarian collaboration has developed 
a device that can reveal whether surgeons are 
slicing through cancerous or normal tissue by 
sniffing and analysing the smoke it produces’. 
“Modern surgery is based on removing all 
the tumour,” explains team member James 
Kinross, a surgeon at Imperial College Lon- 
don. “However, there is still no methodology 
for ensuring this during surgery.’ Estimates 
suggest that more than one in five operations 
intended to remove breast tumours fail to take 
out all malignant cells. 

The ‘intelligent knife’ or iKnife, uses electri- 
cally produced heat to cut the tissue and then 
analyses the smoke produced using mass spec- 
trometry. It identifies the types and concentra- 
tions of the tissue’s metabolites and matches 
the readings to a reference library. This all 
takes less than 3 seconds, in contrast to the 


20-30 minutes needed for histology. As well 
as accurate results, it also shortens the time a 
patient spends under anaesthetic. 

A group headed by Imperial College's Zol- 
tan Takats and Jeremy Nicholson used the 
system in the operating theatre to identify 
in real time 91 pieces of tissue removed from 
patients against a reference database, without 
informing the surgeons of the results. In each 
case, the iKnife’s identification of the type of 
tumour — including tumours from the brain, 
lung, breast, stomach and liver — matched 
those determined by traditional methods. 

The team is now overseeing an observational 
trial in three hospitals involving colonic, breast, 
liver, gynaecological and urological operations. 
“The next phase will be a randomized trial in 
which some surgeons will be able to make deci- 


A tumour viewed by glucoCEST: reds indicate 
higher uptake of glucose compared to blue core. 


sions based on this data and a second group will 
not,’ Kinross says. “We will then be able to deter- 
mine the impact of the device on long-term 
oncological outcomes.’ This study may help to 
address an issue raised by biomedical engineer 
Nimmi Ramanujam of Duke University in Dur- 
ham, North Carolina. Many surgeons, she says, 
would probably prefer to have an image of the 
tumour’s margins before they cut, rather than 
using the iKnife to feel the edges. 


BEAM ME UP 

One of the existing imaging technologies, 
CT, uses X-rays, but it doesn’t have to. Two 
approaches under development rely on more 
exotic forms of radiation: proton beams and 
synchrotron radiation. 

Proton CT records the position, direc- 
tion and energy loss from a proton beam as 
it traverses a patient’s body. Appropriately 
treated, the data produce a three-dimensional 
image of the body that allows physicians to 
diagnose diseases such as cancer and to plan 
treatments. George Coutrakon of Northern 
Illinois University (NIU) in DeKalb explains 
that using protons instead of X-rays provides 
a more detailed image of the body’s density, 
because protons release their energy at pre- 
dictable depths in the body (whereas X-ray 
energy is emitted in a more continuous man- 
ner). This also allows radiologists to target 
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their treatments more precisely, reducing the 
amount of healthy tissue exposed to radiation. 

In 2010, acollaboration between NIU, Loma 
Linda University in California and the Uni- 
versity of California, Santa Cruz, completed a 
prototype proton-beam imaging system. Now 
NIU, the Fermi National Accelerator Labora- 
tory and the Argonne National Laboratory are 
working together to build a second-generation 
device that can produce three-dimensional 
images of something the size of a human head 
in minutes rather than hours. 

While these groups are working with pro- 
tons, an international collaboration is develop- 
ing a CT imaging system based on synchrotron 
X-rays. These are produced when charged 
particles are accelerated around a curved 
path, and have much higher photon ener- 
gies than conventionally generated X-rays. 
Researchers at Ludwig Maximilians University 
in Munich, Germany, and the University of 
California, Los Angeles, have applied a novel 
algorithm to images made by beams at the 
European Synchrotron Radiation Facility in 
Grenoble, France. They have produced three- 
dimensional CT images of the human breast 
with less radiation exposure than for typical 
two-dimensional images. Avoiding high 
exposure to radiation is important because the 
breast is highly radiosensitive. The method is 
still in the fairly early stages of research, but the 
team reports that it could “become a power- 
ful tool for diagnosing breast cancer and allow 
clinicians to battle the disease more effectively.” 

Advances in medical imaging don't always 
happen on such a large scale. Pettigrew cites 
a recent result from an initiative on low-cost 
imaging that his institute started nine years 
ago: the Vscan, a battery-powered, hand-held 
ultrasound device that GE unveiled in 2009. 
“Leading cardiologists hold this as the stetho- 
scope of the future,’ Pettigrew says. “Not only 
is it smaller and fully portable, it costs 20 times 
less than the conventional ultrasound device.” 
Under the same initiative, Rebecca Richards- 
Kortum of Rice University in Houston, Texas, 
is working with Pentax to develop a micro- 
scope small enough to fit into a biopsy needle 
that can produce real-time diagnoses. 

There is so much work underway to develop 
faster, clearer and safer imaging technologies 
that some of the benefits may well be coming 
soon to a clinic near you. “We promote scien- 
tists who develop transformative approaches to 
making new discoveries and acquiring knowl- 
edge about the nature of life and physiology and 
disease,’ says Pettigrew, speaking on behalf of 
his institute and the field of biomedical imaging 
generally. “And we develop new approaches to 
diagnosing and treating disease. = 


Peter Gwynne is a freelance science writer 
based in Sandwich, Massachusetts. 
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